Papers with viable approach
Structured Pruning for Large Language Models Using Coupled Components Elimination and Minor Fine-tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated powerful capabilities in natural language processing, yet their vast number of parameters poses challenges for deployment and inference efficiency. |
| Approach: | They propose a structured pruning algorithm that derives the importance of different components based on intermediate data dependencies and removes coupled components across different layers simultaneously. |
| Outcome: | The proposed algorithm reduces model size and accelerates inference without specialized operators and libraries, while maintaining its utility as versatile problem solvers. |
Improving Tool Retrieval by Leveraging Large Language Models for Query Generation (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown great promise in common sense language understanding, conversational fluency, and reasoning. |
| Approach: | They propose to use Large Language Models to generate a retrieval query and embed it into the prompt to find relevant tools via a nearest-neighbor search. |
| Outcome: | The proposed method improves retrieval for in-domain (seen tools) and out-of-domain settings. |
LAMP-MedQA: A Lightweight Multi-Agent System for Patient-Oriented Medical Question Answering (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) are a promising way to bridge the gap between patient health literacy and access to care. |
| Approach: | They evaluate a range of open- and closed-source LLMs on a MeDiSumQA dataset . they propose a lightweight multi-agent framework for patient-oriented medical question answering . |
| Outcome: | The proposed model achieves lower FKGL than zero-shot GPT-5 and highest simplification quality among all models. |
Word Segmentation as Unsupervised Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing theories of word identification from continuous inputs are based on statistical cues, such as Bayesian inference and normative statistics. |
| Approach: | They propose a model which allows for a process isomorphic to unsupervised constituency parsing and which can reproduce human behavior in word identification experiments. |
| Outcome: | The proposed model reproduces human behavior in word identification experiments, suggesting it is viable to study word identification and its relation to syntactic processing. |
TranSFormer: Slow-Fast Transformer for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prior work has focused on treating subwords as basic units in developing such systems. |
| Approach: | They propose a slow-fast two-stream learning model that uses a “slow” branch to deal with subword sequences and a "fast" branch to cope with longer character sequences. |
| Outcome: | The proposed model shows consistent BLEU improvements (larger than 1 BLUE point) on several machine translation benchmarks. |
Biomedical Event Extraction as Sequence Labeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results show that BeeSL’s speed and accuracy makes it a viable approach for large-scale real-world scenarios. |
| Approach: | They propose a joint end-to-end neural information extraction model that recasts the task as sequence labeling and jointly models intermediate tasks via multi-task learning. |
| Outcome: | Empirical results show that BeeSL outperforms the current best system on the Genia 2011 benchmark by 1.57% absolute F1 score reaching 60.22% F1 . |
MUZO: Leveraging Multiple Queries and Momentum for Zeroth-Order Fine-Tuning of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models incur memory overhead due to the need for activation storage for back-propagation (BP). |
| Approach: | They propose a method that estimates gradients through finite differences without activation storage for back-propagation. |
| Outcome: | The proposed method demonstrates superior performance in fine-tuning various LLMs. |
S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency (2025.naacl-long)
Copied to clipboard
Yuting Zeng, Weizhe Huang, Lei Jiang, Tongxuan Liu, XiTai Jin, Chen Tianying Tiana, Jing Li, Xiaohua Xu
| Challenge: | Large language models exhibit limitations when handling complex mathematical reasoning and logical inference tasks. |
| Approach: | They propose a sparsification strategy to reduce token costs within Multi-agent Debate (MAD) this strategy minimizes ineffective exchanges of information and unproductive discussions among agents . |
| Outcome: | The proposed approach reduces token costs by up to 94.5% while maintaining performance degradation below 2.0%. |
Paraphrase Generation by Learning How to Edit from Samples (2020.acl-main)
Copied to clipboard
| Challenge: | Experimental results show the superiority of our retrieval-based paraphrase generation model in terms of both automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
| Approach: | They propose a retrieval-based method for paraphrase generation which uses a novel editor module to extract edits from paraphrase pairs. |
| Outcome: | The proposed model outperforms existing models in automatic metrics and human evaluation of relevance, grammaticality, and diversity of generated paraphrases. |
Deciphering Stereotypes in Pre-Trained Language Models (2023.emnlp-main)
Copied to clipboard
Weicheng Ma, Henry Scheible, Brian Wang, Goutham Veeramachaneni, Pratim Chowdhary, Alan Sun, Andrew Koulogeorge, Lili Wang, Diyi Yang, Soroush Vosoughi
| Challenge: | Current approaches for examining stereotypes in PLMs require intricate human knowledge about these stereotypes and entail careful manual curation of examples. |
| Approach: | They propose a framework for examining stereotype-encoding behavior of PLMs using model probing and textual analyses. |
| Outcome: | The proposed approach can debiase PLMs without compromising their language modeling capabilities or performance. |
Finetuning Pretrained Transformers into RNNs (2021.emnlp-main)
Copied to clipboard
Jungo Kasai, Hao Peng, Yizhe Zhang, Dani Yogatama, Gabriel Ilharco, Nikolaos Pappas, Yi Mao, Weizhu Chen, Noah A. Smith
| Challenge: | Efficient transformers outperform recurrent neural networks in natural language generation, but this comes with significant computational cost and memory footprint during generation. |
| Approach: | They propose to convert a pretrained transformer into its efficient recurrent counterpart, improving efficiency while maintaining accuracy. |
| Outcome: | The proposed transformers outperform recurrent neural networks in natural language generation but come with significant computational and memory footprint during generation. |
Truth Knows No Language: Evaluating Truthfulness Beyond English (2025.acl-long)
Copied to clipboard
Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria de-Dios-Flores, Rodrigo Agerri
| Challenge: | a new benchmark evaluates the truthfulness of large language models (LLMs) based on imitative falsehoods. |
| Approach: | They propose a professionally translated extension of the TruthfulQA benchmark . it evaluates truthfulness in Basque, Catalan, Galician, and Spanish . |
| Outcome: | The proposed extension of the TruthfulQA benchmark evaluates truthfulness in Basque, Catalan, Galician, and Spanish. |